A modern web application for importing solar power generation data from SolarMan Excel exports into a PostgreSQL database. Features a complete full-stack implementation with Angular frontend and Spring Boot backend.
┌─────────────────┌ REST API ┌──────────────────┌ JDBC ┌─────────────────┌
│ Angular 21 │◄────────────────►│ Spring Boot │◄───────────►│ PostgreSQL │
│ Frontend │ │ Backend │ │ Database │
│ (Port: 4200) │ │ (Port: 8080) │ │ (Port: 5432) │
└─────────────────┘ └──────────────────┘ └────────┌────────┌
│
│ Read-Only
│
┌────────▼────────┌
│ Grafana │
│ Monitoring │
│ (Port: 3000) │
└─────────────────┌
When deployed to Kubernetes, init containers ensure proper startup order:
- PostgreSQL starts first (no dependencies)
- Backend & Grafana wait for PostgreSQL to be ready (port 5432)
- Frontend waits for Backend to be ready (port 8080)
This prevents connection errors and ensures clean application startup.
- 🌐 Modern Web UI: Angular 21 with Material Design and Routing
- 📊 Production Visualization: Interactive bar chart showing last 7 days of solar production
- ⚡ Tshwane Usage Visualization: Bar chart showing kWh consumed between the last 7 Tshwane meter readings
- 🔄 Time-Weighted Calculations: Accurate energy production metrics matching Grafana dashboards
- 🧭 Multi-Page Navigation: Dedicated Home and Upload pages with toolbar navigation
- 📊 Real-time Status: Live database connectivity monitoring with 10-second polling
- 🔄 Auto-Refresh Charts: Charts automatically update after data imports
- 📂 File Upload: Drag-and-drop Excel file processing
- 👀 Data Preview: Review data before importing with Material tables
- 🚀 Full File Import: Processes thousands of records efficiently
- 📈 Import Results: Detailed statistics and error reporting
- 🔄 Dual Support: SolarMan and Tshwane electricity data formats
- 🛡️ Data Validation: Comprehensive file and data validation
- ⚡ High Performance: Optimized for large Excel files
- 🎯 Production Ready: Complete Kubernetes deployment with Rancher Desktop
- 📊 Grafana Integration: Optional monitoring and data visualization
You can run the application in three different ways:
Run the complete application stack in containers with Kubernetes orchestration:
# 1. Build Docker images
./scripts/build-images.sh
# 2. Deploy to Kubernetes
./scripts/k8s-deploy.sh
# 3. Access the application
open http://localhost:30080📘 Full Docker & Kubernetes Documentation →
Run all services with Docker Compose for quick local testing:
# Start all services
./scripts/docker-compose-up.sh
# Access the application
open http://localhost:8081
# Stop services
docker-compose downRun components natively for active development:
- Java 17 or higher
- Node.js 18+ and npm
- PostgreSQL database
- Maven 3.6+
- Trivy (for security scanning)
1. Database Setup
# Start your PostgreSQL database
/Users/danieloots/LOOTS_PG/loots_pg.sh
# Set environment variables
export DB_USER=your_database_username
export DB_PASSWORD=your_database_password2. Backend Setup
# Navigate to backend directory
cd backend
# Build and run Spring Boot application
mvn spring-boot:run
# Backend runs on http://localhost:80803. Frontend Setup
# Navigate to frontend directory
cd frontend/solarman-ui
# Install dependencies and start development server
npm install
npm start
# Frontend runs on http://localhost:42004. Access the Application
Open your browser to http://localhost:4200 and start importing your Excel files!
This project includes comprehensive documentation organized into specialized files:
- AGENTS.md — Main project documentation (overview, features, setup, API reference, deployment, troubleshooting)
- frontend/solarman-ui/README.md — Frontend component structure, Angular conventions, and TypeScript models
- CHANGELOG_v1.7.md — Version history, release notes and migration guide
- grafana/README.md — Grafana dashboard documentation and backup/restore guide
- backend/SECURITY.md — Security scanning with Trivy (quick reference + full guide)
- backend/src/test/README.md — Backend and frontend unit test documentation
- ✅ New
TshwaneChartComponent: Green CSS bar chart on home page between Solar Production and System Status - ✅ kWh per interval: Shows
cumulative_electricity_useddifference between each of the last 7 readings - ✅ New API endpoint
GET /api/database/tshwane-usageusing LATERAL join query - ✅ Auto-refresh: Reloads after Tshwane import via
ChartRefreshService - ✅ 61 backend and 42 frontend tests passing
- ✅ New column
cumulative_electricity_used: Stores running total from Col C of "Elektrisiteit Lesings" sheet (replacing raw meter reading) - ✅ Notes from Col O: Sparse milestone notes (e.g. "Prepaid Electricity Installed") now captured in
reading_notes - ✅
reading_amountdropped: Column was always 0, now removed from schema and model - ✅ Upload preview updated: "Cumulative Electricity Used" column shown in data preview
- ✅ All 56 backend and 31 frontend tests passing
- ✅ Tomcat 10.1.54: Upgraded from 10.1.52 (CRITICAL CVE-2026-29145 auth bypass + 6 other CVEs fixed)
- ✅ Spring Framework 6.2.17: Forced override to fix CVE-2026-22737
- ✅ commons-lang3 3.18.0: Forced override to fix CVE-2025-48924
- ✅ Angular 21.2.10: Upgraded from 21.1.x (XSS in i18n fixed)
- ✅ vite & undici: Updated via @angular/build 21.2.8 (path traversal, HTTP smuggling fixed)
- ✅ Zero frontend vulnerabilities: All 18 npm audit issues resolved
- ✅ Java 17: Upgraded from Java 11 to Java 17
- ✅ Angular 21: Upgraded from Angular 20.3 to Angular 21
- ✅ Vitest Migration: Migrated frontend tests from Karma/Jasmine to Vitest
- ✅ Spring Boot 3.5.10: Upgraded from 3.2.2
- ✅ Apache POI 5.5.1: Upgraded from 4.1.1
- ✅ PostgreSQL JDBC 42.7.10: Upgraded from 42.7.3
- ✅ Tomcat 10.1.52: Upgraded from 10.1.35
- ✅ Security Scanning: Integrated Trivy vulnerability scanning for dependencies and Docker images
- ✅ Grafana Backup/Restore: Comprehensive backup and restore system for dashboards and datasources
- ✅ Tomcat Security Update: Upgraded to version 10.1.35 (CVE-2025-24813 fixed)
- ✅ Simplified Dockerfile: Added
Dockerfile.simplefor runtime-only builds - ✅ Enhanced Documentation: New security guides and Grafana backup procedures
- ✅ Production Chart: CSS-based bar chart showing 7-day solar production trends
- ✅ Multi-Page Routing: Separated Home and Upload pages with Angular Router
- ✅ Time-Weighted Stats: New API endpoint with Grafana-compatible calculations
- ✅ Auto-Refresh: Charts automatically update after successful imports
- ✅ Toolbar Navigation: Material Design navigation with active route highlighting
- ✅ Kubernetes Deployment: Complete containerization with Rancher Desktop
- ✅ Enhanced Architecture: Service-based chart refresh and event system
- ✅ Complete Rewrite: Transformed from CLI to full web application
- ✅ Full File Import: Fixed critical issue - now imports ALL records (1,988+) instead of just preview data
- ✅ Smart File Storage: Temporary file storage with unique IDs for reliable full imports
- ✅ Enhanced UI: Real-time record counts, improved preview experience
- ✅ Flexible Date Parsing: Handles multiple date formats automatically
- ✅ Production Ready: Complete CI/CD pipeline with comprehensive testing
- File Processing: Successfully handles large Excel files (tested with 1,988 records)
- Memory Management: Efficient temporary file storage with automatic cleanup
- Error Recovery: Robust error handling with detailed logging and user feedback
- Performance: Optimized database operations with batch processing and UPSERT logic
- User Experience: Intuitive workflow with clear progress indicators and validation
- Angular 21.2: Latest framework with standalone components
- Angular Material: Modern Material Design components
- TypeScript 5.9: Type-safe development environment
- RxJS: Reactive programming for API communication
- Vitest: Fast unit test runner (migrated from Karma/Jasmine)
- SCSS: Advanced styling with component encapsulation
- Spring Boot 3.5.10: Enterprise-grade Java framework
- Spring Data JPA: Powerful ORM with PostgreSQL integration
- Apache POI 5.5.1: Excel file processing and validation
- Apache Tomcat 10.1.54: Embedded web server (security-patched)
- Java 17: Runtime environment
- Maven: Dependency management and build automation
- Hibernate: Advanced database operations and caching
- Trivy: Security vulnerability scanning
- PostgreSQL: Robust relational database with persistent storage
- Docker: Multi-stage builds for optimized images
- Kubernetes: Production-ready orchestration with Rancher Desktop
- nginx: High-performance reverse proxy for frontend
- Grafana: Analytics and monitoring platform with PostgreSQL datasource
- Environment Configuration: Flexible deployment options
- Production Power, Consumption Power, Grid Power
- Battery operations (Charging, Discharging, SoC)
- Feed-in tariff data and purchasing power
- Timestamp validation and filtering (post-2020 data)
- Cumulative electricity used (running total since baseline reading)
- Milestone notes (e.g. prepaid meter installation, infrastructure changes)
- Municipal electricity usage tracking
# 1. Start Database
/Users/danieloots/LOOTS_PG/loots_pg.sh
# 2. Backend Development (Terminal 1)
cd backend
mvn spring-boot:run
# 3. Frontend Development (Terminal 2)
cd frontend/solarman-ui
npm start
# 4. Production Build
npm run build # Frontend
mvn clean package # Backend# 1. Build all images
./scripts/build-images.sh
# 2. Run with Docker Compose
./scripts/docker-compose-up.sh
# 3. View logs
docker-compose logs -f
# 4. Stop services
docker-compose down# 1. Build images
./scripts/build-images.sh
# 2. Deploy to Kubernetes
./scripts/k8s-deploy.sh
# 3. View logs
kubectl logs -l app=backend -f
# 4. Clean up
./scripts/k8s-delete.shDeploy the complete application stack to Kubernetes with proper orchestration:
# Build Docker images
./scripts/build-images.sh
# Deploy to Kubernetes
./scripts/k8s-deploy.sh
# Access application at http://localhost:30080For detailed deployment instructions, see AGENTS.md
Deploy with Docker Compose for simpler environments:
# Start all services
./scripts/docker-compose-up.sh
# Access application at http://localhost:8081The application builds into a single executable JAR file containing both frontend and backend:
# Build production application
cd frontend/solarman-ui && npm run build
cd ../../backend && mvn clean package
# Deploy single JAR
java -jar target/solarman-ui-backend-1.7.0.jar
# Access application at http://localhost:8080- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is proprietary software. All rights reserved.
For technical support or questions:
- Check the relevant documentation files above
- Review the issue tracker
- Contact the development team
- Built with modern web development best practices
- Optimized for solar energy data management
- Designed for enterprise-scale deployments
- Committed to sustainable energy monitoring
Latest Version: 1.7.3 - Tshwane Electricity Usage Bar Chart
Last Updated: April 30, 2026
Status: Production Ready ✅
The application includes automated security scanning using Trivy:
# Run security scan (integrated with Maven build)
mvn verify
# Run security scan standalone
cd backend && ./security-scan.shSecurity reports are generated in backend/reports/:
- Maven dependencies scan
- JAR artifact scan
- Docker image scan
For detailed security documentation, see:
- backend/SECURITY.md - Security scanning guide (includes quick reference)
The application integrates with Grafana for data visualization:
Dashboards:
- Daily Stats - Last 2 days with hourly heatmaps
- Weekly Stats - ISO week aggregations
- Monthly Stats - Long-term trends
- By Week Number - Seasonal patterns
- Tshwane Daily - Rolling avg electricity consumption vs grid purchased comparison
Access:
kubectl port-forward svc/grafana-service 3000:3000 -n default
open http://localhost:3000Backup & Restore:
# Restore all dashboards
./restore-dashboards-fixed.shFor detailed Grafana documentation, see:
- grafana/README.md - Dashboard documentation and backup/restore guide
- 7-day production bar chart with time-weighted calculations
- Tshwane electricity usage bar chart (kWh between last 7 readings)
- System status panel with real-time database monitoring
- Responsive Material Design interface
- File selection and preview
- Import confirmation workflow
- Detailed import results with statistics
- Toolbar with Home and Upload buttons
- Active route highlighting
- Seamless page transitions